首页 » 知识库 » Hermes + DeepSeek V4 Pro 全自动实现：文本转短视频的工作流

Hermes + DeepSeek V4 Pro 全自动实现：文本转短视频的工作流

1840 字

大伟 2026年5月25日 09:07

周末通过飞书聊天窗口，让Hermes 自己创建了一个工作流。

图片

输入文案，输出一个 1080×1920 的 MP4，全过程中不依赖 GPU、不调 API（LLM 除外）、不装重型软件。在MacOS上，五个步骤调用了五个工具，下面具体介绍一下。

流水线总览

文案 → 口播脚本 → TTS 配音 → 语速微调 → 页面设计 → 逐帧渲染 → 合成输出
       Hermes       Edge TTS    ffmpeg      p5.js     Puppeteer    ffmpeg

Step 1：口播脚本 — Hermes（LLM）

原始文案不能直接读，得改成口语旁白。Hermes 做三件事：

  • 控制字数（~250 汉字 → ~70 秒自然语速）
  • 保留英文术语不翻译（TTS 神经语音能处理中英混合）
  • 切分逻辑段落，每个段落对应视频里的一个视觉场景

这一步没有额外工具，就是 LLM 改写。

Step 2：TTS 配音 — Edge TTS

微软 Edge 浏览器的文本转语音引擎，免费调用，不需要 API key。核心配置：

  • 语音： zh-CN-XiaoxiaoNeural
  • 特点：中英混读自然，音质远超 macOS say
  • 耗时：~250 汉字约 70 秒

一句话切换： hermes config set tts.edge.voice zh-CN-XiaoxiaoNeural ，用完恢复默认即可。

Step 2.5：语速微调 — ffmpeg

Edge TTS 出的是自然讲课节奏，如果想再紧凑一点，用 ffmpeg 的 atempo 滤镜：

ffmpeg -y -i narration.ogg -filter:a "atempo=1.2" -vn narration_1.2x.ogg
倍速 听感 250 字时长
1.0 自然讲课 ~72s
1.2 轻快自然 ~60s
1.5 快速有能量 ~48s

调速之后别忘了重算总帧数（新时长 × 30fps）。

Step 3：页面设计 — p5.js

p5.js 是一个浏览器端的创意编程框架，CDN 引入即可，零安装。这里用它设计竖屏 Slide：

  • 画布：1080×1920（9:16 竖屏）
  • 本工作流提供四套视觉主题可选： Minecraft （像素风） / Terminal （极客命令行） / Minimal （克制专业） / Neon （赛博朋克）
  • 整页同时展示标题+段落，不做逐字动画（动画干扰理解，页面设计才加分）
  • 全程用 RGB 色彩模式（headless Chrome 的 HSB 转换不可靠）

输出是一个自包含的 HTML 文件。

Step 4：逐帧渲染 — Puppeteer

Puppeteer 是 Google 维护的 Node.js 库，驱动 headless Chrome。

  • 加载 Step 3 的 HTML，按 30fps 逐帧截取
  • 用 canvas.toDataURL() 而非截图 API（快 10 倍）
  • 60 秒视频 = 1800 张 PNG 帧，渲染耗时约 2-4 分钟（取决于主题复杂度）

前置条件： npm install puppeteer 。

Step 5：合成输出 — ffmpeg

两步命令：

# 帧序列 → H.264 视频
ffmpeg -y -framerate 30 -i frames/frame_%04d.png \
  -c:v libx264 -pix_fmt yuv420p -preset fast -crf 23 \
  -colorspace bt709 -color_primaries bt709 -color_trc bt709 \
  output/video.mp4

# 混入音频
ffmpeg -y -i output/video.mp4 -i narration_1.2x.ogg \
  -c:v copy -c:a aac -b:a 128k -shortest \
  -map 0:v:0 -map 1:a:0 output/final.mp4

关键： bt709 色彩元数据不能省，否则飞书等平台播放黑屏。

最终产物：一个 1080×1920 的 H.264 + AAC 竖屏 MP4，文件大小 4-10MB。

成本

资源 花费
LLM 改写脚本 你用 Hermes 的正常 token
Edge TTS 免费
ffmpeg 免费开源
p5.js CDN 加载，免费
Puppeteer npm 包，免费
GPU 不需要

除 LLM 外的全链路零成本。

一开始用的是逐字弹入 + Perlin 噪声背景路线，文字在画面里跳舞。后来发现那些动画效果反而降低了信息密度。观看者需要的是一个完整的视觉锚点，不是一个字一个字蹦出来的干扰。

现在已经把方案改为幻灯片式整页展示。每一页标题和正文同时出现，视觉变化来自页面的主题风格。字体、配色、纹理、边框，场景之间只有简单的 crossfade 或者直接 hard cut。

四种视觉主题由四个 drawSlide() 函数实现:

Minecraft 风格用 Perlin 噪声模拟石墙纹理，Zpix 中文像素字体，金色标题，像素边框。像在游戏里看对话框。2.7MB，渲染 22fps。石墙每帧 32,400 个 rect 调用，是唯一有明显性能成本的风格。

Terminal 风格就是黑底绿字终端。Fira Code 等宽字体，带 $ 提示符，正文写成 // 注释。0.5MB，渲染 50fps。压缩率极高因为帧间画面完全一致。只适合开发者内容，但记忆点很深。

Minimal 风格极简克制。Noto Sans SC 字体，高对比度，标题左对齐，短分割线。0.5MB，55fps。适合严肃分析和观点类内容。

Neon 风格赛博朋克。Orbitron 字体配霓虹发光边框，深紫底。0.5MB，55fps。

四个风格共用同一段配音、同一个场景列表、同一个帧速率。20 秒视频 603 帧，并行渲染总时间等于最慢那个（Minecraft 约 27 秒）。

一篇文案，一个 skill 调用。Hermes 自动完成脚本改写、TTS 配音、p5 草图生成、帧渲染、视频编码。用户只需要确认最终效果并且通过聊天调优即可。

附录

以下为 text-to-short-video 的完整 SKILL.md（v1.3.0）适合AI自行阅读

---
name: text-to-short-video
description: "Turn text/copy into short vertical videos: script → TTS narration → themed slide design → headless render → ffmpeg assembly. Zero-cost pipeline (no GPU, no API keys beyond LLM)."
version: 1.3.0
metadata:
  hermes:
    tags: [video, short-video, tts, p5js, puppeteer, ffmpeg, content-creation]
---

Text-to-Short-Video Pipeline

Turn a piece of copy into a narrated short video (9:16 vertical) with zero additional cost beyond the LLM you're already using.

Pipeline

文案 → 口播脚本 → TTS 配音 → 语速微调 → p5.js 页面设计 → Puppeteer 渲染 → ffmpeg 合成 → MP4
         ↓            ↓           ↓              ↓                      ↓               ↓
    Hermes 改写   Edge TTS    ffmpeg atempo  Slide 风格 (主题)    canvas.toDataURL   bt709 + AAC
                 zh-CN 神经    1.2x~1.5x     Minecraft/Terminal     逐帧截图           H.264
                 72s/250字                   /Minimal/Neon                        60s@1.2x
               ←─────── 语速对照 ──────→
               atempo=1.0 (72s)  atempo=1.2 (60s)  atempo=1.5 (48s)

Prerequisites

  • Node.js (for Puppeteer headless rendering)
  • ffmpeg (for video encoding and audio mixing)
  • npm install puppeteer in the project directory

No GPU, no LaTeX, no ComfyUI needed.

Step 1: Script Adaptation

Take the original copy and rewrite it for spoken narration:

  • ~250 Chinese characters → ~70 seconds of natural speech
  • Keep English technical terms as-is (zh-CN neural voices handle mixed text)
  • Break into logical segments for scene timing
  • Each segment becomes one visual scene (one "slide")

Step 2: TTS Narration

Use Edge TTS with a Chinese neural voice:

# One-time config (restore to en-US-AriaNeural after if desired):
hermes config set tts.edge.voice zh-CN-XiaoxiaoNeural

Then call Hermes text_to_speech() with the narration script.

Why Edge TTS Chinese neural:

  • Handles mixed Chinese/English naturally
  • Much higher quality than macOS say
  • Free, no API key needed
  • Duration: ~70s for ~250 Chinese characters at natural pace

Pitfall: Default en-US-AriaNeural skips all Chinese text. Must switch to zh-CN-XiaoxiaoNeural.

Step 2.5: Speed Adjustment

Edge TTS produces natural-speed audio (~250 char → ~72s). For a brisker pace without sounding rushed, use ffmpeg atempo:

# 1.2x speed — slightly faster, still natural
ffmpeg -y -i narration.ogg -filter:a "atempo=1.2" -vn narration_1.2x.ogg

# Recalculate frames: new_duration × 30fps
# e.g. 72.6s → 60.5s → 1815 frames

Rate guide: | atempo | feel | 250-char duration | |--------|------|-------------------| | 1.0 | natural lecture pace | ~72s | | 1.2 | brisk but natural | ~60s | | 1.5 | fast, energetic | ~48s |

Beyond 1.5x starts sounding noticeably accelerated.

IMPORTANT: Changing speed changes total frame count. Update TOTAL in the p5 sketch and frame rendering command to match duration × 30. Also redistribute all scene start / end boundaries proportionally — TTS rarely hits the exact duration you estimated. The workflow is: (1) estimate DUR, write sketch with placeholder scenes, (2) generate TTS, (3) measure actual duration, (4) apply atempo, (5) recalc TOTAL + rebuild SCENES array before running the frame render.

Pitfall: Edge TTS does not support -r for rate control (unlike macOS

Step 3: p5.js Page Design (Slide Style — Recommended)

Create a self-contained HTML file with p5.js (CDN, no install).

Approach Comparison

Approach How it works Best for Verdict
Slide (推荐) 整页同时展示: 标题 + 段落，统一主题 教程、观点、人物、分析 ✅ 设计感强，记忆点深
Animation (旧) 逐字弹入 / 逐行reveal 极少数需要节奏感的场景 ⚠️ 动画干扰理解，不推荐

Visual Tier Guide

Tier Elements Render speed File size (60s) When
Baseline Solid bg + system font + full-page fade ~55fps ~1.5MB Draft/preview
Slide (推荐) Theme bg + Google Font + 整页展示 18-55fps 4-10MB Final render
Enhanced (旧) Perlin noise bg + char bounce + Noto SC ~12fps ~9MB Legacy only

Slide Style: Pick One Theme

Choose one of four themes below and use it for ALL scenes:

Theme Font Vibe Render cost
Minecraft Zpix 中文像素 (本地) 像素石墙、中文像素风 中 (stone texture)
Terminal Fira Code 极客、CLI、开发者 低 (solid bg)
Minimal Noto Sans SC 专业、克制、严肃 最低
Neon Orbitron 赛博朋克、AI前沿 低 (solid bg)

Each theme is a self-contained drawSlide() function. Copy ONE into your sketch.

Slide Structure (no start/end per scene)

const W = 1080, H = 1920;
const FPS = 30;

// Simple scene array — evenly distributed by TOTAL frames
const SCENES = [
  {t:'标题',     s:'副标题 / 引入句',          c:[0,190,230]},
  {t:'要点一',   s:'正文段落。\n可多行。',      c:[255,140,0]},
  {t:'要点二',   s:'正文段落二。',              c:[80,200,120]},
  {t:'收尾',     s:'总结金句。',                c:[0,190,230]},
];

const TOTAL = Math.floor(audioDuration * FPS);
const fps = Math.floor(TOTAL / SCENES.length); // frames per scene

function getCurrentScene(fc) {
  let idx = Math.min(Math.floor(fc / fps), SCENES.length - 1);
  let e = constrain((fc - idx * fps) / fps, 0, 1);
  return {scene: SCENES[idx], ease: e, index: idx};
}

Crossfade or Hard Cut

Add this at the top of your theme's drawSlide() for smooth crossfade:

let alpha = 1;
if (e < 0.1) alpha = map(e, 0, 0.1, 0, 1);
else if (e > 0.9) alpha = map(e, 0.9, 1, 1, 0);
drawingContext.globalAlpha = alpha;

For Minecraft / Terminal themes, a hard cut (no crossfade) often feels more authentic. Skip the alpha block entirely.

CRITICAL: Use RGB color mode, not HSB

Headless Chrome's HSB→RGB conversion is unreliable. Always use RGB:

// WRONG: colorMode(HSB, 360, 100, 100, 100);
// RIGHT: use RGB arrays directly
const BG=[12,16,30];       // dark navy
const WHITE=[245,248,255]; // off-white
const CYAN=[0,190,230];    // accent cyan

Vertical Layout (9:16)

Canvas: 1080×1920. Text sizes vary by theme: General ranges:

  • Titles: 48-84px (depends on font)
  • Body: 28-44px
  • Line height: 1.4-1.6x

Headless Mode Support

The sketch must support headless rendering. For themes using Google Fonts CDN (Minimal, Terminal, Neon), the standard pattern works:

function setup() {
  createCanvas(W, H);
  frameRate(FPS);
  noiseSeed(42);
  window._p5Ready = true;
  if (window._HEADLESS) noLoop();
}

For themes using locally-served custom fonts via CSS @font-face (Minecraft/Zpix), set _p5Ready AFTER the font loads, not at end of setup():

function setup() {
  createCanvas(W, H);
  frameRate(FPS);
  noiseSeed(42);
  // MUST wait for font before signaling ready
  document.fonts.load('60px Zpix').then(() => {
    window._p5Ready = true;
    if (window._HEADLESS) noLoop();
  }).catch(() => {
    window._p5Ready = true;  // fallback
    if (window._HEADLESS) noLoop();
  });
}

Using document.fonts.ready instead of explicit .load('60px Family') will cause silent render failures (pitfall #16). The font-family and pixel size in .load() must match what drawSlide() actually uses.

Step 4: Headless Frame Rendering

Use Puppeteer with canvas.toDataURL() — NOT the screenshot API. Two render scripts are provided:

  • scripts/export-frames.js — uses file:// protocol (fastest, but fonts may fail for locally-served TTF)
  • scripts/render-http.js — uses http://localhost:8765 (needed for CSS @font-face with local TTF files). Start server first: python3 -m http.server 8765
# For Google Fonts CDN themes (Minimal, Terminal, Neon):
node scripts/export-frames.js sketch_terminal.html frames_terminal 603

# For local font themes (Minecraft/Zpix):
python3 -m http.server 8765 &
node scripts/render-http.js sketch_minecraft.html frames_minecraft 603

**Performance:** 
- Baseline (solid bg): ~55fps in headless
- Slide themes: 18-55fps depending on bg complexity (Minecraft stone texture slowest,
  Terminal/Minimal fastest)
- File size: 1.5-10MB depending on theme and frame-to-frame variation

## Step 5: Video Encoding + Audio Mixing

\`\`\`bash
# Encode frames to video (with bt709 color metadata)
ffmpeg -y -framerate 30 -i output/frames/frame_%04d.png \
  -c:v libx264 -pix_fmt yuv420p -preset fast -crf 23 \
  -colorspace bt709 -color_primaries bt709 -color_trc bt709 \
  output/video.mp4

# Mix with narration audio (use speed-adjusted audio if applicable)
ffmpeg -y -i output/video.mp4 -i narration_1.2x.ogg \
  -c:v copy -c:a aac -b:a 128k -shortest \
  -map 0:v:0 -map 1:a:0 output/final.mp4

CRITICAL: Include -colorspace bt709 -color_primaries bt709 -color_trc bt709 or the video will render as black on Feishu and other messaging platforms.

Pitfalls (from real debugging sessions)

  1. HSB color mode → black frames in headless Chrome. Always use RGB arrays.
  2. Edge TTS en-US voice skips Chinese. Switch to zh-CN-XiaoxiaoNeural.
  3. macOS say Tingting voice has broken -r parameter. Rate 300 = 17.8x speedup instead of 1.6x. Use Shelley or Edge TTS instead.
  4. Puppeteer screenshot API times out on many frames. Use canvas.toDataURL() (10x faster).
  5. Missing color metadata → black video on Feishu. Add bt709 flags to ffmpeg.
  6. Thin strokes invisible at 1080p. Use solid filled shapes (rect, circle) instead of thin lines.
  7. p5Ready must be set at END of setup(), not at top-level — p5.js initialization order matters.
  8. Google Fonts / CSS @font-face fonts NOT accessible via p5 textFont() in headless Chrome. p5's textFont('Family') can't use CSS-loaded fonts for canvas rendering — always falls back to default. Working solution: use native Canvas 2D drawingContext.font = '42px Family' + drawingContext.fillText() instead. Set _p5Ready only after document.fonts.load('42px Family') resolves (NOT document.fonts.ready — that's insufficient). For CJK pixel fonts, use Zpix (最像素, github.com/SolidZORO/zpix-pixel-font). Press Start 2P is Latin-only (no Chinese glyphs).
  9. Press Start 2P body text too wide. The pixel font has very wide characters. For body text with this font, keep lines to ~18 characters max. Use text() with maxW parameter for auto-wrapping.
  10. Can't visually verify rendered frames on models without vision. Providers like deepseek don't support image input — vision_analyze fails with unknown variant 'image_url'. You cannot inspect rendered frames to confirm content is correct. Workaround: before the long render, triple-check that the narration script text and the sketch's SCENES array match. If user disputes content after delivery, show them the script file, verify MD5 hashes differ from any prior video, and re-send. Don't trust that the render "must be right" — you can't see it.
  11. TTS actual duration ≠ estimate — always post-adjust scene timings. The workflow is: (1) estimate DUR with placeholder TOTAL, write sketch, (2) generate TTS, (3) measure actual duration with ffprobe, (4) apply atempo → new duration, (5) recalculate TOTAL = floor(new_duration × 30), (6) rebuild fps and scene distribution, (7) THEN run the frame render. Skipping (6) produces frames that don't match the audio pacing.
  12. MEDIA send race condition — user reports "wrong content". ffmpeg may return exit code 0 before the OS finishes flushing the output file to disk. Sending via MEDIA tag immediately can deliver a partial/broken file that the user can't play, leading to complaints like "内容不对" or "视频发错了吗". Before sending: (a) run ffprobe on the final MP4 to verify both video and audio streams exist with expected duration, (b) run ls -la to confirm file size is in the expected range (5-12MB for 40-60s), (c) compute MD5 to confirm it differs from any prior output. If user still reports wrong content after sending, don't argue — re-send immediately. The second send almost always works because the file has fully flushed by then.
  13. Press Start 2P is Latin-only — Chinese text renders as □/?. This pixel font has no CJK glyphs. For Chinese content, use a Chinese pixel font like Zpix (最像素, github.com/SolidZORO/zpix-pixel-font). Prefer CSS @font-face for large CJK fonts — p5 loadFont() parses the entire TTF via opentype.js, which can timeout on fonts >1MB.
  14. Malformed HTML → silent render timeout with no visible error. Duplicate <head> / <body> tags or <script> inside <script> prevent p5.js from calling setup(). The render script hangs for 30s then throws TimeoutError: Waiting failed: 30000ms exceeded on _p5Ready. The page doesn't throw any JS error — p5 simply never initializes.Before rendering, validate:single <head>, single <body>, no <script> tag leakage, font links inside <head> not <body>. The file:// protocol makes these failures silent and hard to diagnose.
  15. Zpix (7.2MB) / large CJK fonts time out p5 loadFont(). opentype.js parses the entire TTF, stalling for 120s+ on fonts >1MB. Workaround: use CSS @font-face + native Canvas 2D for text (pitfall #8), which lets the browser handle font loading. For p5 loadFont specifically, subset the font to needed characters with fontTools — a 94-char subset of Zpix goes from 7.2MB to 12KB.
  16. document.fonts.ready is insufficient for CSS font loading. The Promise resolves before fonts are actually usable in headless Chrome. Always use explicit document.fonts.load('42px Family') and set _p5Ready in its .then() callback — not at end of setup(). The sketch's renderFrame() will fail silently if you call it before the font resolves.
  17. rect() in CORNER mode with CENTER coordinates → off-screen rendering. Default rectMode(CORNER) means rect(x, y, w, h) positions by top-left. Code like rect(W/2, ty, 860, 120) places the left edge at x=540, so an 860px-wide rect extends to x=1400 on a 1080px canvas — half off-screen with no warning. Fix: use explicit top-left coords — rect((W-w)/2, ty-h/2, w, h). The border rects in the Minecraft theme are written correctly for CORNER mode; only title/body boxes with center-intended coords are affected.

AI与创业 · 目录

继续滑动看下一个

AgenticAI前哨站

向上滑动看下一个

💬 讨论 (0) 发起新帖 ← 返回知识库